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ABSTRACT 



This report discusses the performance of students with 
disabilities on the Kentucky Instructional Results Information System (KIRIS) 
on-demand assessments. These assessments, which consist of constructed 
response or essay-like questions over the period of 1992-93 through 1995-96, 
are part of the regular KIRIS assessment, which also includes writing and 
mathematics portfolio entries. Results from the data indicate student 
performance is not just improving, but perhaps improving at a rate more rapid 
than that of general education students, at least in certain grades. Also 
evident are very notable grade level differences in student participation. 
Approximately 9 percent of students taking the KIRIS in Grade 4 and 8 percent 
of students taking the KIRIS in Grade 8 are students with disabilities, while 
in Grade 11/12, only about 4 percent of the students in the assessment are 
students with disabilities. The percentage of students with disabilities 
taking the test with accommodations is quite high, ranging from a low of 
about 62 percent in Grade 11/12 to a high of about 84 percent in Grade 4. 
Finally, the performance levels of students with disabilities on KIRIS were 
below those of students in the general population, with the gap between 
performance levels more noticeable at higher grade levels. (CR) 



******************************************************************************** 



* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 

******************************************************************************** 



O 

ERIC 



State Assessment Series 

Maryland/Kentuck y 

7 l.^TV . ^ . Report 3 
K>,;W./ 




"'NV'rr 




>0 

\ 

o- 

\_5- 

O 

- CO; 
,5J 

w. 

ERIC 



Performamtlf&sSna Use 
of Accommodations on a 
Statewide Assessment 



t ■}' 'i 'i\i?;.:V f _ - 






State Assessment Series 

Maryland/Kentucky 



Report 3 



Performance Trends and Use 
of Accommodations on a 
Statewide Assessment 

Students with Disabilities in the KIRIS 
On-Demand Assessments from 1992-93 
through 1995-96 



The Maryland-Kentucky-NCEO Assessment Project 



Prepared By: 



Scott Trimble 

Kentucky Department of Education 



The Maryland-Kentucky-NCEO Assessment Project encompasses a 
comprehensive array of research studies on assessment accommodations 
and alternate assessment approaches that facilitate inclusion of all stu- 
dents in statewide assessment programs. The planned research program 
include: 

• comparative studies involving a qualitative examination of the assess- 
ment systems in Kentucky and Maryland; 

• studies that involve secondary analyses of existing data bases to 
address critical technical and implementation issues in the assessment of 
students with disabilities; and 

• experimental field studies that involve the collection of new data to 
address several technical issues crucial to the development and modifica- 
tion of state assessment policies. 



This project is supported, in part, by a grant to the Maryland State 
Department of Education from the U.S. Department of Education, 
Office of Special Education Programs (Grant #H023F50008). 
Opinions expressed herein do not necessarily reflect those of the U.S. 
Department of Education or Offices within it. 



Permission is granted to copy this document without charge. Additional 
print copies may be purchased for $10.00 from: 

NATIONAL CENTER ON EDUCATIONAL OUTCOMES 

University of Minnesota • 350 Elliott Hall 
75 East River Road • Minneapolis, MN 55455 
612.624.8561 • Fax 612.624.0879 
http://www.coled.umn.edu/NCEO 



This document is available in alternative formats upon request. 



Overview 



3 



State Assessments Pushing Educational Reform 

State assessments and accountability systems are among the principle approaches to educational 
reform. Despite varying opinions about the approach, policymakers have agreed about the need 
for these assessments and accountability systems to be inclusive of all children. When exclusion 
and exemptions from assessments and accountability systems are allowed, numerous problems 
emerge and questionable practices often occur. For example, both increases in retention rates 
and rates of referral to special education were documented when exclusion from high stakes 
assessments was allowed (Allington & McGill-Franzen, 1992; Zlatos, 1994). 

Students who receive special education services, in fact, are the most likely students to be 
excluded from state assessments. And, sometimes, even when they have taken the assessments, 
these students have been excluded from the accountability systems (e.g., their scores have been 
excluded when scores are reported, or their scores were not considered when decisions were 
made about rewards and sanctions). In other words, schools and school districts are not being 
held accountable for the performance of these students. These policies matter in that as educators 
consider strategies to improve their standing within an accountability system, students with 
disabilities and other exempted students can be, and are, ignored in decisions about how resources 
are re-distributed in personnel and finances. 

Changes in Special Education Law 

The traditional practice of exclusion of students with disabilities from state and district-wide 
assessments is no longer possible if states want to continue to receive their special education 
funds from the federal government. In June, 1997, amendments to the Individuals with Disabilities 
Education Act (IDEA) were enacted that require, among other things, states to: (a) include 
students with disabilities in their regular assessments, with accommodations where appropriate, 
(b) report the number of students with disabilities participating in the regular state assessment, 
and (c) report the performance of students with disabilities on these assessments in the same 
way and with the same frequency as they are reported for other students. Furthermore, for those 
students unable to take the regular assessment, states must develop and implement an alternate 
assessment by the year 2000, and also report on the performance of students on these assessments 
in 2000. 

Part of the reason for these dramatic requirements in IDEA is the lack of data on the performance 
of students with disabilities, particularly in relation to educational standards. Individualized 
Educational Programs (IEPs), which are required for each student receiving special education 



services, have not provided the information needed to evaluate the performance of students 
with disabilities. Nor have IEPs provided the information needed to systematically evaluate the 
programs serving these students, particularly with regard to how well the programs are enabling 
students with disabilities to progress toward the same standards expected of all students. 

Kentucky’s Inclusive Accountability System 

Despite the overall lack of information on the performance of students with disabilities in relation 
to state standards, there have been some states that have been striving to include students with 
disabilities and to gather data on their performance. Chief among these is the Commonwealth 
of Kentucky. Since 1990, when it incorporated the philosophy that all students can learn, and 
that the educational system needs to be accountable for the learning of all students, Kentucky 
has moved toward a totally inclusive assessment and accountability system. 

From the time of the establishment of the Kentucky Instructional Results Information System 
(KIRIS), Kentucky has had a policy of including all students. This policy encompassed those 
students considered to be at risk of failure, and those students with legally identified disabilities, 
which included both students with formal IEPs and students with 504 plans. To enable the 
greatest number of students possible to participate in the regular KIRIS assessments, Kentucky 
established a comprehensive policy on assessment accommodations. Students were allowed to 
use, during assessment, any accommodation that they used during instruction. 

Kentucky’s policy of inclusiveness in assessments extended to inclusion of all students in school 
and district accountability. Thus, the scores of students with disabilities who participated in 
KIRIS assessments counted in the same way as the scores of students without disabilities. 

As a result of its policies, Kentucky has a wealth of information on the performance of students 
with disabilities on a “regular state assessment.” While some researchers have pulled bits of 
information from this comprehensive set of data, for example, for certain years of testing (Koretz, 
1997), no one has yet reported on the complete set of the KIRIS on-demand assessment data. 
Particularly lacking are studies describing performance trends over time, an important aspect of 
the reform-driven system in Kentucky. 

The purpose of this report is to take a significant step in that direction — to provide a more 
comprehensive picture of how students with disabilities are performing over time on the KIRIS 
on-demand assessments. These assessments, which consist of constructed response or essay- 
like questions over the period 1992-93 through 1995-96, are a significant part of the regular 
KIRIS assessment, which also includes writing and mathematics portfolio entries. Data from 
the on-demand assessments reflect the performance of more than 99% of Kentucky’s student 
population in the grades assessed (Grades 4, 8, and 11/12). 
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It should be noted that Kentucky was unique among states in the early 1990s in that it realized 
that only a very small percentage of students should be considered unable to participate in the 
regular assessment, even with the use of accommodations. These students were ones who 
generally were unable to participate in the general curriculum and who were not pursuing a 
high school diploma (less than 1% of Kentucky’s school population), a determination made by 
the Admissions and Release Committee (an IEP team). Because Kentucky wanted to be 
accountable for the learning of all students, it developed an Alternate Portfolio Assessment for 
these students. Scores from this assessment have an impact equal to those of the assessments of 
other students in the school accountability process (see Ysseldyke, Thurlow, Erickson, Haigh, 
Moody, Trimble, & Insko, 1997). (Information on the performance of students on the Alternate 
Portfolio Assessment is presented in a separate report.) 

Thus, students with disabilities in Kentucky were included in KIRIS in one of three ways: 

• Participate in the regular KIRIS assessment components in the same way as nearly all 
other students. 

• Participate in the KIRIS Assessment components using accommodations that have been 
a part of the student’s instructional program during the year. 

• Participate in the KIRIS Alternate Portfolio Assessment, if eligible according to specific 
criteria (see Table 1). 

As noted previously, the focus of this report is the performance of those students with disabilities 
who participate in the regular on-demand assessment activities in one of two ways: 

• With no accommodations to the assessment administration (because they do not receive 
instructional accommodations) 

• With instructional accommodations applied to the assessment administration (because 
they receive these accommodations during instruction) 

In this report, we also look at the performance of students with disabilities using various types 
of accommodations. 
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Table 1. Eligibility Criteria for Participation in Kentucky’s Alternate Portfolio Assessment 



(a) The student’s demonstrated cognitive ability and adaptive behavior itself 
prevents completing the course of study even with program modifications 

(b) The student’s current adaptive behavior requires extensive direct instruction 
in multiple settings to accomplish the application and transfer of skills 
necessary for functional application in domestic, community living, 
recreational/leisure, and vocational activities in school, work, home, and 
community environments 

(c) The student’s inability to complete the course of study may not be the result 
of excessive or extended absences; it may not be primarily the result of 
visual or auditory disabilities, specific learning disabilities, emotional- 
behavioral disabilities, social, cultural, or economic differences 

(d) The student is unable to apply or use academic skills at a minimal 
competency level in natural settings (such as the home, community, or work 
site) when instructed solely or primarily through school based instruction 

(e) For eighth and twelfth grade students with disabilities, the student is unable 
to 

(1) Complete a regular diploma program even with extended school 
services, schooling, program modifications, and adaptations 

(2) Acquire, maintain, generalize skills and demonstrate performance 
without intensive frequent and individualized community-based 
instruction 

NOTE. These criteria are taken from a May 1992 Kentucky Department of 

Education Program Advisory. All criteria must be met for a student to be eligible 

for the Alternate Portfolio Assessment. 
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Participation Rates and Frequency of Accommodations 



Approximately 4% to 10% of Kentucky’s students, depending on grade level and year, are 
identified as having disabilities when participating in the KIRIS assessments. These percentages, 
which are shown in Table 2 according to grade and year, are based on data coded on the KIRIS 
answer documents at the time of testing. Because of the nature of this coding and because data 
are collected in April, it is possible that some students with disabilities are not coded, and thus 
the percentages of students with disabilities appearing in the table may be different from those 
reported to the Office of Special Education Programs (OSEP) each year. The December child 
counts provided to OSEP are by age, while Kentucky’s assessment data are dependent on spring 
grade placement. Thus, direct comparison of numbers is not possible. 

Given the available data, we note that the percentage of students with disabilities at the fourth 
grade level tended to increase some over the four year period, from 8.73% of those assessed 
using the regular KIRIS assessment components in 1992-93, to 10.18% of those assessed in 
1995-96. At the eighth grade level, the percentage remained rather constant, varying between 
7.21% and 8.44%. At the 1 1 th/ 1 2th grade levels, the percentage of students with disabilities 
identified as participating in the regular KIRIS increased from 3.71% in 1992-93 to 4.95% in 
1995-96. 

The use of accommodations during assessments also is documented on the KIRIS answer 
documents by local district staff. The numbers and percentages of students using accommodations 
during the 1994-95 and 1995-96 assessments are shown in Table 3. This information was not 
specifically tagged to students in the data collection process prior to the 1994-95 assessment. 

During the two years for which data were available, the percentage of the total population 
taking the KIRIS assessment components with accommodations remained relatively constant. 
At the fourth grade level, approximately 8% to 9% of the total student population used 
accommodations; at the eighth grade level, approximately 5.5% used accommodations; and, at 
the eleventh/twelfth grade level, approximately 3% of the total student population used 
accommodations . 

Of those students whose tests were coded to indicate that they had a disability, approximately 
82% to 84% of fourth graders took the assessment with accommodations during 1994-95 and 
1995-96. Approximately 68% to 69% of eighth graders, and approximately 62% at the eleventh/ 
twelfth graders with disabilities used accommodations during these years. 

Accommodations used during KIRIS must be consistent with those accommodations that students 
receive during instruction. For example, if students normally have printed materials read to 
them during the school year, the printed materials associated with the KIRIS assessments (e.g., 
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Table 2. Numbers and Percentages of Students Participating in KIRIS On-Demand 
Assessments 



Grade/Year 


Students w Disabilities 
Number Percent 


General Students 
Number Percent 


Total Number 
Students 


GRADE 4 


1992-93 


4322 


8.73 


45187 


91.27 


49509 


1993-94 


4647 


9.33 


45139 


90.67 


49786 


1994-95 


4884 


9.88 


44541 


90.12 


49425 


1995-96 


4864 


10.18 


42895 


89.82 


47759 


GRADE 8 


1992-93 


3564 


7.21 


45852 


92.79 


49416 


1993-94 


4383 


8.44 


47557 


91.56 


51940 


1994-95 


4075 


7.96 


47123 


92.04 


51198 


1995-96 


4083 


8.05 


46609 


91.95 


50692 


GRADE 11/12 


1992-93 


1336 


3.71 


34710 


96.29 


36046 


1993-94 


1710 


4.67 


34910 


95.33 


36620 


1994-95 


2017 


4.88 


39286 


95.12 


41303 


1995-96 


2080 


4.96 


39844 


95.04 


41924 



Note: General Students = students without disabilities 

In 1995-96, 546 students were miscoded. They were not included in either population. 
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Table 3. Numbers and Percentages of Students Using Accommodations During KIRIS On- 
Demand Assessments 



Grade/Year 


Number 


Percent 1 


GRADE 4 


1994-95 


4006 


8.11 


1995-96 


4118 


8.62 


GRADE 8 


1994-95 


2778 


5.43 


1995-96 


2808 


5.54 


GRADE 11 


1994-95 


1242 


3.01 


1995-96 


1288 


3.07 



’Percent refers to the number of students with disabilities using accommodations divided by the total 
number of students (see Table 1). 

constructed response items or portfolio related activities) may be read to these students. If the 
normal instructional process provides for the students to be able to dictate responses to a scribe, 
such students may dictate responses to items administered for the KIRIS assessments. It is 
intended that these instructional accommodations be consistent with best instructional practices 
considering each child’s disability and individual needs. 
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Trends in the Performance of Students with Disabilities on KIRIS 



Kentucky’s policy of including students with disabilities in its assessments emphasizes the 
importance of monitoring the performance of students with disabilities. The overriding purpose 
of the inclusive policy is to assure that there be continued emphasis on improvement in 
instructional programs, and that the improvement reaches all components of the student 
population. Furthermore, the instructional improvement must be reflected in improvement in 
students’ achievement, and it must be shown for all students. 

This analysis of student performance is focused on the reading, mathematics, science, and social 
studies KIRIS on-demand assessments administered during the second accountability cycle 
(1992-93 through 1995-96). Also administered during this cycle were portfolio assessments in 
mathematics and writing (as well as the Alternate Portfolio Assessment mentioned previously). 
Data from the on-demand writing assessment are not included here. During Kentucky’s second 
accountability cycle (ending in 1996), these data were not yet included in accountability 
calculations. Similarly, data from two other components of the accountability system (arts and 
humanities; practical living/vocational studies) are scaled differently, and not included in these 
analyses. 

Although KIRIS performance is reported in terms of four performance standards (novice, 
apprentice, proficient, and distinguished) in each content area assessed, the standards are based 
on an underlying scale score that is derived from the application of a two parameter graded- 
response IRT scaling procedure applied to a 7-item test (constructed response items) at the 
fourth grade and an 8-item test at the eighth and eleventh grades. (Note that in the 1992-93 and 
1993-94 school years, the on-demand assessments were administered at the 12th grade; they 
were moved to 11th grade in 1994-95. Also, the 1992-93 and 1993-94 eighth and twelfth grade 
assessments were seven open-response items.) 

In the base year 1992-93, the scale scores in each of the four content areas had a mean of zero 
(0) and a standard deviation of one (1). For the following three years (1993-94 through 1995- 
96), this scale has been equated to reflect the changes in student performance over the 4-year 
period. The findings are presented here for each grade level. 

Grade 4 

In Table 4, we summarize the performance of students with disabilities in terms of the four 
performance standards (novice, apprentice, proficient, and distinguished) for the four content 
areas of reading, mathematics, science, and social studies during the 1995-96 testing. Clearly, 
all students are demonstrating performance at the lower end of the standards (novice and 
apprentice). W hi le the percentage of students with disabilities in the proficient and distinguished 
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Table 4. Numbers and Percentages of Grade 4 Students Scoring within each Performance 
Standard during 1995-96 Testing 



Content/ Students w Disabilities General Students Total Population 

Standard Number Percent Number Percent Number Percent 



Reading 


Novice 


593 


12 


2495 


6 


49509 


7 


Apprentice 


3303 


62 


25820 


63 


49786 


63 


Proficient 


1385 


26 


12549 


30 


49425 


30 


Distinguished 


64 


1 


532 


1 


47759 


1 


Mathematics 


Novice 


2088 


40 


11795 


29 


13883 


30 


Apprentice 


2524 


47 


23674 


58 


26198 


56 


Proficient 


430 


8 


3727 


9 


4157 


9 


Distinguished 


303 


6 


2200 


5 


2503 


5 


Science 


Novice 


2276 


44 


12194 


29 


14570 


31 


Apprentice 


2939 


55 


27758 


67 


30697 


66 


Proficient 


125 


2 


1406 


3 


1531 


3 


Distinguished 


5 


0 


38 


0 


43 


0 


Social Studies 


Novice 


1925 


37 


11949 


29 


13874 


29 


Apprentice 


2770 


53 


24254 


59 


27024 


57 


Proficient 


611 


11 


5017 


12 


5628 


12 


Distinguished 


39 


1 


176 


0 


215 


1 



Note: Total percentages may not equal 100 due to rounding. 
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levels is comparable to the percentage of general students in these levels, more students with 
disabilities than general students tend to fall into the novice category, and more general students 
than students with disabilities tend to fall into the apprentice category. 

In Figure 1, we summarize the actual equated scale scores of students with disabilities and 
general students (i.e., students without disabilities) in Grade 4 on the reading, mathematics, 
science, and social studies assessments over the four year period from 1992-93 to 1995-96. In 
general, in this grade the performance gap was narrowed between students with disabilities and 
the general population of students without disabilities. For example, in reading (where the mean 
equated scale scores for both groups of students continued to increase over the four year period), 
students with disabilities began with an equated scale score of -.60, compared to .03 for general 
students, while in 1995-96, students with disabilities scored .63, compared to .83 for general 
students. 

An exception to the tendency to narrow the gap might be in science where, in 1994-95, there 



Figure 1. Summary of Equated Scale Scores of Students with Disabilities and General 
Students in Grade 4 on the Reading, Mathematics, Science, and Social Studies Assessments, 
1992-93-1995-96 




YEAR 



YEAR 



-RDNG Disabled 
-RDNG General 





YEAR 



YEAR 



YEAR 



-MATH Disabled 
- MATH General 




-SOC Disabled 
- SOC General 
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was a .05 difference between the two groups as compared to .25 in 1995-96. However, this 
difference is still smaller than that observed in the 1992-93 school year where the mean for 
general students exceeded the mean for students with disabilities by .65. 

It is of interest to note that while Grade 4 students in the general population experienced very 
modest declines in the 1995-96 school year in mathematics, science, and social studies, students 
with disabilities experienced modest growth in mathematics and social studies. At the same 
time, the .27 drop in science scores for students with disabilities was the largest noted. 

In Figures 2-5, we have plotted the mean equated scale scores for each content area summarized 
in Figure 1, along with the points 1 standard deviation above and below these means. These 
figures include data for Grade 4 students with disabilities, students in the general population 
without disabilities, and then the total population of students, including both students with and 
without disabilities. These figures allow for the observation of the overall impact of including 
students with disabilities in the assessments on total score distributions. The data from which 
these figures were constructed are included in Appendix A. 



Figure 2. Reading Equated Scale Scores for Grade 4 Students with Disabilities, Students in the 
General Population Without Disabilities, and Total Population of Students 
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Figure 3. Mathematics Equated Scale Scores for Grade 4 Students with Disabilities, Students 
in the General Population Without Disabilities, and Total Population of Students 
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Figure 4.Science Equated Scale Scores for Grade 4 Students with Disabilities, Students in the 
General Population Without Disabilities, and Total Population of Students 
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Figure 5. Social Studies Equated Scale Scores for Grade 4 Students with Disabilities, Students 
in the General Population Without Disabilities, and Total Population of Students 
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Figures 2-5 demonstrate that the inclusion of test scores from students with disabilities in the 
overall score distribution has only a small negative impact. And, as the performance of students 
with disabilities approached the performance of students in the general population, this impact 
was even smaller. The data in these figures also indicate that, in general, the gap between the 
performance of students with disabilities in Grade 4 and the performance of the general population 
of students is closing. 

Grade 8 

In Table 5, we summarize the performance of students with disabilities, general students, and 
the total population in terms of the four performance standards (novice, apprentice, proficient, 
and distinguished) for reading, mathematics, science, and social studies during the 1995-96 
testing. As in Grade 4, most students are demonstrating performance in the lower performance 
standards (novice and apprentice). Unlike Grade 4, the percentages of students with disabilities 
in the proficient and distinguished levels are not comparable to the percentage of general students 
in these levels, except perhaps in science where minimal numbers of students demonstrated 
performance at the proficient and distinguished levels. Like Grade 4, more students with 
disabilities than general students performed at the novice category, and more general students 
than students with disabilities performed at the apprentice level. 
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Table 5. Numbers and Percentages of Grade 8 Students Scoring within each Performance 
Standard during 1995-96 Testing 



Content/ Students w Disabilities General Students Total Population 

Standard Number Percent Number Percent Number Percent 



Readin g 



Novice 


1059 


25 


1847 


4 


2906 


6 


Apprentice 


3157 


74 


37515 


82 


40672 


81 


Proficient 


50 


i 


6357 


14 


6407 


13 


Distinguished 


0 


0 


40 


0 


40 


0 


Mathematics 


Novice 


3003 


70 


15051 


33 


18054 


36 


Apprentice 


966 


23 


17022 


38 


17988 


36 


Proficient 


182 


4 


7587 


17 


7769 


16 


Distinguished 


115 


3 


6099 


13 


6214 


12 


Science 


Novice 


3631 


85 


23657 


52 


27288 


55 


Apprentice 


632 


15 


21976 


48 


22608 


46 


Proficient 


3 


0 


125 


0 


128 


0 


Distinguished 


0 


0 


1 


0 


1 


0 


Social Studies 


Novice 


2920 


69 


12136 


27 


15056 


30 


Apprentice 


1258 


29 


27501 


60 


28759 


58 


Proficient 


83 


2 


5705 


13 


5788 


12 


Distinguished 


5 


0 


417 


1 


422 


1 



Note: Total percentages may not equal 100 due to rounding. 
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In Figure 6, we have summarized the performance of Grade 8 students with disabilities and 
general students over the four years, 1992-93 to 1995-96. It is evident in these figures that the 
performance gap was narrowed between students with disabilities in Grade 8 and the general 
population of students without disabilities. For example, in reading (where both groups of students 
continued to grow over the four year period), students with disabilities began with an equated 
scale score of -1.14, compared to .02 for general students, while in 1995-96, students with 
disabilities scored -.44, compared to .38 for general students. In other words, the difference in 
scores of students with disabilities and general students went from 1.15 standard deviation 
units in the base year to approximately 0.82 standard deviation units in 1995-96. 



Figure 6. Summary of Equated Scale Scores of Students with Disabilities and General 
Students in Grade 8 on the Reading, Mathematics, Science, and Social Studies Assessments, 
1992 - 93 - 1995-96 
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Students in the general population performed basically the same in 1995-96 as in 1994-95 in 
reading and mathematics, and experienced declines in the 1995-96 school year in science and 
social studies. Students with disabilities also experienced a modest decline in science 
performance, but their performance remained relatively flat in the other content areas. It is also 
noteworthy that while the gap between the performance of the two groups of students continued 
to narrow over time, the gap is noticeably larger than that observed at the Grade 4 level. In fact, 
the smallest gaps at Grade 8 just barely approach the largest gaps at Grade 4. 

In Figures 7-10, we have plotted the mean equated scale scores for each content area summarized 
in Figure 6, along with the points 1 standard deviation above and below these means. As in the 
figures for Grade 4 (Figures 2-5), these figures also provide for the comparison of the general 
population of students without disabilities with the total population including both students 
with and without disabilities. The data from which these figures were constructed are included 
in Appendix B. Because students with disabilities in Grade 8 have not closed the gap between 
their performance and that of the students in the general population to the extent that appears to 
have happened at the Grade 4 level, the impact of the scores of students with disabilities on the 
total distribution is slightly more noticeable. 



Figure 7. Reading Equated Scale Scores for Grade 8 Students with Disabilities, Students in the 
General Population Without Disabilities, and Total Population 
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Figure 8. Mathematics Equated Scale Scores for Grade 8 Students with Disabilities, Students 
in the General Population Without Disabilities, and Total Population 
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Figure 9. Science Equated Scale Scores for Grade 8 Students with Disabilities, Students in the 
General Population Without Disabilities, and Total Population 
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Figure 10. Social Studies Equated Scale Scores for Grade 8 Students with Disabilities, 
Students in the General Population Without Disabilities, and Total Population of Students 
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Grade 11 

In Table 6, we summarize the performance of students with disabilities, general students, and 
the total population in terms of the four performance standards (novice, apprentice, proficient, 
and distinguished) for reading, mathematics, science, and social studies during the 1995-96 
testing. As for Grades 4 and 8, most students are demonstrating performance at the novice and 
apprentice levels. Similar to Grade 8, the percentages of students with disabilities in the proficient 
and distinguished levels are not comparable to the percentage of general students in these levels. 
And, like both Grades 4 and 8, more students with disabilities than general students performed 
at the novice level, and more general students than students with disabilities performed at the 
apprentice level. 

Figure 11 is a summary of Grade 11 performance on the reading, mathematics, science, and 
social studies assessments from 1992-93 to 1995-96. Because it is generally accepted that the 
Grade 11 students taking the KIRIS in the 1992-93 school year were somewhat less motivated 
than in the following years, performance in the 1992-93 school year at the high school level is 
difficult to interpret. Therefore, it may be more useful to focus on the three year pattern (1993- 
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Table 6. Numbers and Percentages of Grade 11 Students Scoring within each Performance 
Standard during 1995-96 Testing 



Content/ Students w Disabilities General Students Total Population 

Standard Number Percent Number Percent Number Percent 



Readin g 



Novice 


1787 


81 


10869 


28 


12656 


31 


Apprentice 


390 


18 


23940 


62 


24330 


59 


Proficient 


9 


0 


3810 


10 


3819 


9 


Distinguished 


0 


0 


92 


0 


92 


0 



Mathematics 



Novice 


1671 


77 


11637 


30 


13344 


32 


Apprentice 


479 


22 


17458 


46 


17937 


44 


Proficient 


24 


1 


6618 


17 


6642 


16 


Distinguished 


12 


1 


2998 


8 


3010 


7 



Science 



Novice 


1217 


56 


4541 


11 


5758 


13 


Apprentice 


962 


44 


30025 


77 


30987 


76 


Proficient 


6 


0 


3786 


10 


3792 


9 


Distinguished 


1 


0 


359 


1 


360 


1 



Social Studies 



Novice 


1714 


79 


12250 


32 


13964 


34 


Apprentice 


457 


21 


21151 


55 


21608 


53 


Proficient 


15 


1 


4897 


13 


4912 


12 


Distinguished 


0 


0 


413 


1 


413 


1 



Note: Total percentages may not equal 100 due to rounding. 
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Figure 11. Summary of Equated Scale Scores of Students with Disabilities and General 
Students in Grade 11 on the Reading, Mathematics, Science, and Social Studies Assessments, 
1992 - 93 - 1995-96 





94 to 1995-96). As was evident in the Grade 4 and Grade 8 data, students with disabilities in 
Grade 1 1 narrowed the gap between their performance and that of the general population of 
students without disabilities. In reading, the population of students with disabilities obtained an 
equated scale score of -1.00 in 1993-94, compared to a value of .47 for general students, while 
in 1995-96, students with disabilities scored -.63, compared to .46 for general students. (If the 
four year period were considered, the comparable beginning points for the students with 
disabilities would be -1.38, compared to -.04 for general students.) 

The relative distance between performance of students with disabilities is noticeably larger 
than that observed at the Grade 4 level, and in the 1995-96 school year, somewhat larger than 
that observed at the Grade 8 level. The performance of both groups remained relatively flat 
over the 1994-95 and 1995-96 years, with the exception that students with disabilities showed 
a modest increase in 1995-96, and general students experienced a modest decline in social 
studies. 
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Figures 12-15 are plots of the mean equated scale scores for each content area summarized in 
Figure 11, along with the points 1 standard deviation above and below these means. These 
figures also provide for the comparison of the general population of students without disabilities 
with the total population, including both students with and without disabilities. Because the 
students with disabilities in Grade 1 1 had not closed the gap between their performance and that 
of the general population, particularly when compared to changes noted in Grade 4, the impact 
on the total distribution is slightly more noticeable. The data from which these figures were 
constructed are included in Appendix C. 



Trends in Performance with Accommodations ■ * • * * 

The accommodations used by students with disabilities during KIRIS can be grouped into eight 
categories: 

• None. No accommodations; the student with disabilities takes the assessment under the 
same conditions as all other students. 

Figure 12. Reading Equated Scale Scores for Grade 11 Students with Disabilities, Students in the 
General Population Without Disabilities, and Total Population of Students, Reading Assessment 
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Figure 13. Mathematics Equated Scale Scores for Grade 11 Students with Disabilities, Students 
in the General Population Without Disabilities, and Total Population of Students 
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Figure 14. Science Equated Scale Scores for Grade 11 Students with Disabilities, Students in the 
General Population Without Disabilities, and Total Population of Students 
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Figure 15. Social Studies Equated Scale Scores for Grade 11 Students with Disabilities, 
Students in the General Population Without Disabilities, and Total Population of Students 
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• Reader/Oral. The assessment is read to the student in a way comparable to the manner 
in which normal instruction is delivered. 

• Scribe/Dictation. The responses to the assessment are dictated by the student and written 
down by a scribe in a way comparable to the manner in which normal instruction is 
delivered. 

• Cueing. The student with disabilities uses mnemonics, problem-solving organizers, 
semantic organizers, webs, or templates in responding to the assessment in a way 
consistent with daily instruction and the student’s IEP or 504 Plan. 

• Paraphrasing. The content of the assessment is paraphrased for the student with 
disabilities in a way comparable to the manner in which normal instruction is delivered. 

• Interpreter. The content of the assessment is signed for the student with disabilities in 
a way comparable to the manner in which normal instruction is delivered. 

• Technological. Technology typically used by the student with disabilities in daily 
instruction is made available during the administration of the assessment. 
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• Other. Other accommodations normally made available in the delivery of instruction 
are made available in the administration of the assessment. 

Although the policy permitting use of accommodations has been in place since 1992, collection 
of data on accommodations used during assessment did not begin until 1994-95. Because students 
could use those accommodations that they used during instruction, an array of combinations of 
accommodations was possible. For the analyses presented here, only those accommodations 
and combinations of accommodations used by a least 100 students were included. The specific 
accommodations and combinations that met this criterion varied by grade and assessment year, 
as shown in Table 7. 

The most frequently used accommodations were combinations that included paraphrasing and 
oral presentations. Of the eight primary categories of accommodations, four were not used 
during the assessment by at least 100 students: cueing, interpreter, technological, and other. The 
only accommodations or combinations used by at least 100 students across all grades and both 
years were paraphrasing, and paraphrasing & oral. 

Grade 4 

In Figures 16-19, we have plotted the 1994-95 and 1995-96 mean equated scale scores (along 
with the points 1 standard deviation above and below these means) for (a) the total population 
of students in Grade 4, (b) those Grade 4 students who received any accommodation or 
combination of accommodations (if used by at least 100 students), separately for each 
accommodation and combination, and (c) those Grade 4 students who used no accommodations. 
A separate figure is used for each content area. The data from which these figures were constructed 
are included in Appendix D. 

Variability in the effects of accommodations is evident in each of the Grade 4 figures. There 
were two accommodations/combinations in each year that resulted in mean performance above 
that of the total population. In 1994-95, the oral & dictation combination, and the paraphrasing 
& oral & other combination, produced mean scores above the total. In 1995-96, dictation, and 
the paraphrasing & dictation & other combination, produced mean scores above the total. Another 
observation that is evident in these figures is that some accommodations do not result in higher 
scores than those obtained by students with disabilities using no accommodations, but in fact, 
seem to be associated with lower performance. This is particularly evident for the oral 
accommodation, where in all cases except one, the mean equated scale score is lower for those 
students using the accommodation than for students with disabilities using no accommodations. 
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Table 7. Accommodations and Combinations of Accommodations Included in Analyses 
(n >100) 





Grade 4 
1994-95 1995-96 


Grade 8 
1994-95 1995-96 


Grade 11/12 
1994-95 1995-96 


Reader/Oral (Oral) 


524 


375 


597 


507 


(dOO) 


161 


Scribe/Dictation(Dic) 


(<100) 


122 


(dOO) 


(dOO) 


(dOO) 


(dOO) 


Paraphrasing (Para) 


189 


222 


308 


356 


205 


270 


Oral & Die 


857 


907 


(dOO) 


160 


(dOO) 


(dOO) 


Para & Oral 


717 


517 


1105 


1023 


553 


559 


Para & Oral & Die 


(<100) 


1620 


327 


413 


(dOO) 


(dOO) 


Para & Oral & Other 


1344 


(<100) 


(dOO) 


(dOO) 


(dOO) 


(dOO) 


Para & Oral & 


(<100) 


105 


(dOO) 


(dOO) 


(dOO) 


(dOO) 


Tech & Die 














None 


1007 


746 


1420 


1275 


845 


792 



Note: Entries are numbers of students using the accommodation. 

Grade 8 



Mean equated scale scores (along with the points 1 standard deviation above and below these 
means) for Grade 8 students for each of the content areas are plotted in Figures 20-23. Again 
data are plotted for the total population of students, for each accommodation or combination of 
accommodations (used by at least 100 students), and for students who used no accommodations 
during the two testing years. The data from which these figures were constructed are included 
in Appendix D. 

The effectiveness of various accommodations or combinations is much less clear in the Grade 
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Figure 16. Reading Equated Scale Scores for Grade 4 Students, Showing Total Population 
Students Who Used Accommodations, and Students Who Used No Accommodations 




Figure 17. Mathematics Equated Scale Scores for Grade 4 Students, Showing Total Population, 
Students Who Used Accommodations, and Students Who Used No Accommodations 
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Figure 18. Science Equated Scale Scores for Grade 4 Students, Showing Total Population, 
Students Who Used Accommodations, and Students Who Used No Accommodations 
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Figure 19. Social Studies Equated Scale Scores for Grade 4 Students, Showing Total Population, 
Students Who Used Accommodations, and Students Who Used No Accommodations 
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8 data than it was in the Grade 4 data. The highest scoring group of students with disabilities in 
reading was the one using the paraphrasing & oral combination. However, in 1994-95 this 
group obtained a mean equated scale score of -.23, while the total population obtained a mean 
of .29. In 1995-96, the corresponding values were -.14 for students with disabilities using this 
combination, and .32 for the total population. There were no accommodations or combinations 
of accommodations in either year that resulted in mean performance above that of the total 
population. 

Grade 11 

In Figures 24-27, we have plotted the 1994-95 and 1995-96 mean equated scale scores (along 
with the points 1 standard deviation above and below these means) for (a) the total population 
of students in Grade 11, (b) those Grade 11 students who received any accommodation or 
combination of accommodations (if used by at least 100 students), separately for each 
accommodation and combination, and (c) those Grade 1 1 students who used no accommodations. 
A separate figure is used for each content area. The data from which these figures were constructed 
are included in Appendix D. 



Figure 20. Reading Equated Scale Scores for Grade 8 Students, Showing Total Population, 
Students Who Used Accommodations, and Students Who Used No Accommodations 
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Figure 21. Mathematics Equated Scale Scores for Grade 8 Students, Showing Total Population, 
Students Who Used Accommodations, and Students Who Used No Accommodations 




Figure 22. Science Equated Scale Scores for Grade 8 Students, Showing Total Population, 
Students Who Used Accommodations, and Students Who Used No Accommodations 
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Figure 23. Social Studies Equated Scale Scores for Grade 8 Students, Showing Total Population, 
Students Who Used Accommodations, and Students Who Used No Accommodations 
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At Grade 1 1 , the highest scoring group of students with disabilities in reading was that receiving 
the paraphrasing & oral combination. In 1994-95, this group obtained a mean equated scale 
score of -.80 while the total population obtained a mean of .42. In 1995-96, the corresponding 
values were -.63 for students with disabilities and .37 for the total population. As in Grade 8, 
there were no accommodations or combinations of accommodations in either year that resulted 
in mean performance above that of the total population. In fact, on the equated scale score 
metric, at both Grades 8 and 11 (where at least 100 cases were reported), the means for students 
with disabilities, with or without accommodations, were noticeably below the zero point, while 
the total population (including students with disabilities) is noticeably above the zero point. 



Summary 

Given the complexity and variability of findings in the data presented in Figures 16 - 27, it is 
somewhat difficult to draw out general trends. Clearly, the accommodation or combination that 
resulted in higher performance varied some depending on the content area, grade, and year. 
However, the pattern of accommodations being more effective (i.e., resulting in higher scores) 
in Grade 4 (compared to Grades 8 and 11) seemed to be rather stable within the 1994-95 and 
1995-96 data. However, this observation must be qualified in a number of ways because of the 
differences in the populations of students with disabilities in Grades 4, 8, and 11. 
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Figure 24. Reading Equated Scale Scores for Grade 11 Students, Showing Total Population, 
Students Who Used Accommodations, and Students Who Used No Accommodations 
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Figure 25. Mathematics Equated Scale Scores for Grade 11 Students, Showing Total Population, 
Students Who Used Accommodations, and Students Who Used No Accommodations 
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Figure 26. Science Equated Scale Scores for Grade 11 Students, Showing Total Population, 
Students Who Used Accommodations, and Students Who Used No Accommodations 
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Figure 27. Social Studies Equated Scale Scores for Grade 11 Students, Showing Total 
Population, Students Who Used Accommodations, and Students Who Used No 
Accommodations 
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As previously noted (see Table 3), the percent of students with disabilities using accommodations 
was somewhat different from the elementary to the high school levels. In the 1994-95 school 
year, 8.11% of the total population was students with disabilities using accommodations, while 
9.88% of the total population was students with disabilities overall. Thus, 1.77% of the total 
population consisted of students with disabilities who used no accommodations during the 
assessment. In Grade 8, 5.43% of the total population was students with disabilities who used 
accommodations, while 2.53% of the total population was students with disabilities not using 
any accommodations. In Grade 11, 3.01% were students with disabilities using accommodations, 
while 1.77% were students with disabilities not using any accommodations. Another important 
consideration when attempting to decipher the findings in Figures 16 - 27 is the decline in the 
percent of the total population that is identified as students with disabilities as the grade level 
increases. 



Conclusions and Implications ■» ■ < ■ > j ~ 1 

The Kentucky performance data provide one of the first consistent sets of data with a longitudinal 
perspective and an inclusive approach to assessment and accountability. The data also provide 
some of the first information on the effects of the use of various accommodations on performance. 
Overall, these data present a picture of student performance that is not just improving, but 
perhaps improving at a rate more rapid than that of general education students, at least in certain 
grades. The data also indicate that while some information is available on the use of 
accommodations and the differences in performance that seem to be related to their use, there is 
need for further information on questions related to the effects of accommodation on performance. 
There is also evidence of the evolving nature of the system within the data. For example, there 
has been a steady increase in the percentage of students with disabilities assessed using the 
regular KIRIS from 1992-93 to 1995-96. Yet, the use of accommodations has remained relatively 
stable. 

Also evident are very notable grade level differences, both in the numbers of students with 
disabilities taking the KIRIS on-demand assessments, and in the percentage of students using 
accommodations. Thus, we see that approximately 9% of students taking the KIRIS in Grade 4 
and 8% of students taking the KIRIS in Grade 8 are students with disabilities, while in Grade 
11/12, only about 4% of the students in the assessment are students with disabilities. 

One might speculate that as students get older, decisionmakers view the regular assessment as 
less appropriate for students with disabilities, and therefore, the decision makers are more likely 
to move students into the alternate assessment system. However, this seems inconsistent with 
the Alternate Portfolio data. While there was an increase from the initial year (1993) in all 



grades (4, 8, and 12), the number of students participating in the Alternate Portfolio at the 12th 
grade for the four year period (1993-1996) has remained relatively stable (189, 241, 201, and 
219, respectively, for the four years). Of course, the change in numbers of students with disabilities 
participating in KIRIS also might reflect the decrease in students receiving special education 
services at these higher grades, as well as the possibility that students drop out of school when 
they reach the high school level. 

Also of note, and consistent with the findings of Koretz (1997), the percentage of students with 
disabilities taking the test with accommodations is quite high, ranging from a low of about 62% 
in Grade 11/12 to a high of about 84% in Grade 4. Although we do not have a good way to 
estimate the percentages of students with disabilities for whom we would expect that 
accommodations would be needed, these percentages might be viewed as high. In other states 
(e.g., Maryland, Rhode Island), the percentage of students using accommodations seems to be 
lower. Any comparisons of states, however, are complicated by differences in the nature of the 
assessment, which might play a significant role in determining the need for accommodations 
during the assessment. Furthermore, the question of most relevance, and one that can easily get 
lost when the focus is on data, is an instructional question. When teaching students with 
disabilities “on grade level” challenging content, and evaluating the student’s ability to apply or 
make use of such content, what instructional accommodations, and in what magnitude, are 
appropriate? 

The performance levels of students with disabilities on KIRIS, regardless of grade level or year 
of testing, were below those of students in the general population. Using the equated scale 
scores for comparisons, the performance of students with disabilities was generally considerably 
below that of other students. The gap between the performance of the two groups of students 
was more noticeable at the higher grade levels. 

Despite the lower performance of students with disabilities, the impact of their lower scores 
was limited, as suggested by visual inspection of the figures showing performance of the total 
group, performance of general students, and performance of students with disabilities. Because 
students with disabilities accounted for less than 10% of the total population, the impact of their 
scores on the mean scores was marginal. On the Grade 4 reading test in 1993, for example, the 
mean test score dropped less than one-tenth of a standard deviation unit when the test scores of 
students with disabilities were merged with the scores of their peers without disabilities. In 
subsequent years, the change was even smaller. 

Of interest, but not all that surprising, was the finding that the spread of the scores grew when 
the scores of students with disabilities were combined with the scores of their peers without 
disabilities. Other states and local education agencies are likely to have similar results. Therefore, 
district superintendents and state departments of education need not be concerned that their 
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student population scores will “look bad” when all students are included in their reports. Thus, 
state and local summary data can be made complete and inclusive of the total student population. 

In all grades, there was a trend of increasing performance over time, at a rate generally exceeding 
that of the general population of students. The achievement gap between Grade 4 students with 
disabilities and their peers in the general population decreased dramatically over the four years. 
In reading, the gap was 0.6 equated scale score units in 1993, and only 0.2 units in 1996. 
Student growth over the four years was equally as dramatic. In reading, students with disabilities 
gained an average of 1.2 equated scale score units; in math, approximately 1.0 equated scale 
score units; approximately 0.9 in science, and 0.75 in social studies. In other words, the gap 
between students with disabilities and students in the general population seems to be closing 
over time. This is an extremely important finding because of the concern that educational reforms 
do not always benefit all students. In fact, Kentucky’s reforms were designed with this particular 
concern in mind. In part, this concern was the one impetus for including all students in the 
accountability system. The findings suggest that it is working, and doing so even though many 
other specific instructional approaches have failed to narrow the performance gap between 
general education and special education students. 

It should be noted that the impact of the statewide implementation of an ungraded primary 
program may also have had an impact on performance and performance trends. This is a matter 
for further study. Although peers in the general population did not show as much gain, their 
improvement could also be called dramatic. In reading, the difference between general students 
who were in Grade 4 in 1996 and those in Grade 4 in 1993 was a positive 0.8 units. In math and 
science, the difference was a positive 0.5 equated scale score units, and in social studies, the 
difference was approximately .25 units. The only drop in mean achievement scores was for 
students with disabilities on the science test between 1995 and 1996 (-0.25), but over the four 
year period, growth was still evident. 

The data on the effects of specific accommodations are somewhat more difficult to interpret. As 
noted previously, student performance with accommodations generally is still lower than the 
performance of general students. Only in 4 of the 104 cases examined (oral & dictation, 1994- 
95; paraphrasing & dictation & other, 1994-95; dictation, 1995-96; paraphrasing & dictation & 
other, 1995-96) was the performance of students using accommodations higher than the 
performance of the total group of students. The fact that paraphrasing and dictation appear in 
these cases suggests the need for further examination of these accommodations. However, the 
fact that the unexpected difference in performance occurred so rarely may suggest an anomaly 
in findings. 

It is important to note that it is difficult to draw specific conclusions about the meaning of the 
higher performance since we do not have the experimental basis that we need to reach certain 
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conclusions. For example, there is no comparison group of students without disabilities receiving 
the accommodations, a condition that is needed to actually demonstrate that the accommodations 
may be raising performance in unacceptable ways. Another important comparison might be 
students with disabilities all instructed with appropriate accommodations, but assessed with 
and without these accommodations. 

The assumption behind the use of accommodations is that they should reveal students’ true 
performance levels, levels that are obstructed from being demonstrated because of the disability. 
Without the accommodations, we are getting a measure of the effect of the student’s disability. 
We would expect that by circumventing the effect of the disability, performance should be 
higher. Simply finding higher performance levels does not mean that the accommodation is 
giving the student an advantage. It may simply be a demonstration that the effects of the disability 
have been bypassed, and that true levels of performance are evident. Experimental designs are 
needed to answer these questions. 

The finding that in some cases performance with accommodations was lower than performance 
with no accommodations might suggest that incorrect decisions are being made about the best 
accommodations for students to use. This may be an incorrect conclusion to reach, however, 
since we do not know how those students who used the accommodation would have performed 
without it. Furthermore, the finding may reflect an interaction of decision making with how 
resources to provide instructional accommodations are distributed across students with different 
disabilities. For example, only those with the most severe needs may be targeted to receive 
accommodations. Again, experimental designs are needed to answer these types of questions. 

There are additional factors to consider as well. For example, there has been little consideration 
or study of how the various instructional/assessment accommodations interact with the curriculum 
at the elementary, middle, and high school levels. In that the curriculum at the elementary level 
might be thought of as more basic in nature, the accommodations typically applied to both 
instruction and assessment may be more effective than at upper grade levels where the curriculum 
might be thought of as more complex in both content considerations and applications. At the 
elementary level, a student typically is in contact with one or two teachers most of the day 
across the total curriculum. This might create an environment where it is relatively easy to 
understand and implement instructional accommodations consistently across the content. At 
the middle and high school levels, students typically are engaged with five or more teachers 
across the curriculum. In this environment, it may be more difficult to coordinate and deliver 
appropriate instructional accommodations. 

For all students with disabilities, the amount of time spent exposed to the daily regular curriculum 
delivered in a mainstream classroom setting may have an impact on the consistent delivery of 
instructional accommodations, and therefore, their depth of knowledge in content areas. Another 
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matter of inquiry might focus on an analysis of the actual level of instruction experienced by 
students with disabilities at the middle and high school levels, as compared to the level or 
complexity of instruction delivered to the general population of students at the middle school 
level. 

Based on observations resulting from routine interaction with schools as opposed to a well 
designed random process of inquiry, it appears that some students with disabilities at the middle 
and high school levels are being instructed at levels below what is normal for the general 
population. If confirmed to be a systematic problem, this practice would reasonably result in 
the need for fewer accommodations because the student may be attacking curriculum where 
such accommodations are not needed. This practice would also result in the population of students 
with disabilities less likely to have engaged instruction exposing them to the higher content and 
application expectations typically encountered in the KIRIS assessments. 

Certainly these are only several possible hypotheses, and there are numerous others that might 
be proposed. Obviously, much work remains to be done to understand the performance of students 
with disabilities in statewide assessments. This study, however, is an important first step in 
doing so, particularly since it provides a longitudinal look at performance. 
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Mean Equated Scale Scores, Standard Deviations, and Counts for Grades 4, 8, and 
11: Data Used in Figures 1 through 27 
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Appendix A. Grade 4 Students with Disabilities Compared to General Population: Mean Equated 
Scale Score, Standard Deviations, and Counts 



YEAR 


POPULATION 


MEAN 


READING 

+1SD -1SD 


ST.DEV. 




MEAN 


MATHEMATICS 

+ 1SD -1SD 


ST.DEV. 


1993 


Disabled 


-0.6026 


0.3398 


-1.5450 


0.9424 




-0.6466 


0.2765 


-1.5698 


0.9231 


1993 


General 


0.0278 


0.9624 


-0.9068 


0.9346 




0.0396 


0.9535 


-0.8743 


0.9139 


1993 


Total 


-0.0273 


0.9248 


-0.9793 


0.9520 




-0.0203 


0.9147 


-0.9553 


0.9350 


1994 


Disabled 


-0.1261 


0.7981 


-1.0504 


0.9242 




0.1033 


0.8871 


-0.6805 


0.7838 


1994 


General 


0.3127 


1.1136 


-0.4883 


0.8010 




0.4197 


1.1920 


-0.3526 


0.7723 


1994 


Total 


0.2717 


1.0949 


-0.5515 


0.8232 




0.3902 


1.1690 


-0.3887 


0.7788 


1995 


Disabled 


0.5822 


1.5353 


-0.3710 


0.9532 




0.3580 


1.1895 


-0.4736 


0.8316 


1995 


General 


0.7982 


1.6501 


-0.0538 


0.8520 




0.6112 


1.3217 


-0.0994 


0.7106 


1995 


Total 


0.7768 


1.6417 


-0.0881 


0.8649 




0.5861 


1.3135 


-0.1412 


0.7274 


1996 


Disabled 


0.6262 


1.5378 


-0.2854 


0.9116 




0.3890 


1.1573 


-0.3793 


0.7683 


1996 


General 


0.8301 


1 .6568 


0.0035 


0.8266 




0.5704 


1.2529 


-0.1122 


0.6825 


1996 


Total 


0.8094 


1.6473 


-0.0286 


0.8380 




0.5519 


1.2458 


-0.1420 


0.6939 








SCIENCE 








SOCIAL STUDIES 








MEAN 


+1SD 


-1SD 


ST.DEV. 




MEAN 


+1SD 


.-1SD 


ST.DEV. 


1993 


Disabled 


-0.6246 


0.4223 


-1.6715 


1 .0469 




-0.6326 


0.4327 


-1.6979 


1.0653 


1993 


General 


0.0279 


0.9118 


-0.8561 


0.8839 




0.0267 


0.9303 


-0.8768 


0.9036 


1993 


Total 


-0.0291 


0.8889 


-0.9471 


0.9180 




-0.0308 


0.9067 


-0.9683 


0.9375 


1994 


Disabled 


-0.2131 


0.7187 


-1.1449 


0.9318 




-0.2775 


0.7928 


-1.3478 


1.0703 


1994 


General 


0.1992 


1.0163 


-0.6179 


0.8171 




0.1468 


1.0763 


-0.7827 


0.9295 


1994 


Total 


0.1607 


0.9978 


-0.6764 


0.8371 




0.1072 


1.0588 


-0.8444 


0.9516 


1995 


Disabled 


0.5234 


1.4067 


-0.3600 


0.8834 




0.0393 


1.1763 


-1.0978 


1.1370 


1995 


General 


0.5736 


1.3293 


-0.1821 


0.7557 




0.3093 


1.3109 


-0.6923 


1.0016 


1995 


Total 


0.5686 


1 .3380 


-0.2007 


0.7694 




0.2826 


1.3015 


-0.7363 


1.0189 


1996 


Disabled 


0.2585 


1.0413 


-0.5242 


0.7828 




0.1286 


1.0333 


-0.7761 


0.9047 


1996 


General 


0.5076 


1.2304 


-0.2153 


0.7228 




0.2605 


1.1027 


-0.5816 


0.8422 


1996 


Total 


0.4822 


1.2153 


-0.2508 


0.7330 




0.2471 


1.0967 


-0.6026 


0.8497 




NUMER PERCENT 



1993 


Disabled 


4322 


8.73% 


1993 


General 


45187 


91.27% 


1993 


Total 


49509 


100.00% 


1994 


Disabled 


4647 


9.33% 


1994 


General 


45139 


90.67% 


1994 


Total 


49786 


100.00% 


1995 


Disabled 


4884 


9.88% 


1995 


General 


44541 


90.12% 


1995 


Total 


49425 


100.00% 


1996 


Disabled 


4864 


10.18% 


1996 


General 


42895 


89.82% 


1996 


Total 


47759 


100.00% 



MEAN = Mean 4th grade equated scale score. 

+1SD = Mean plus one standard deviation. 

-1SD = Mean minus one standard deviation. 

ST.DEV. = Standard deviation of equated scale score. 
N-Counts are constant across content within each year. 
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Appendix B. Grade 8 Students with Disabilities Compared to General Population: Mean Equated 
Scale Score, Standard Deviations, and Counts 



YEAR 


POPULATION 




READING 








MATHEMATICS 








MEAN 


+1SD 


-1SD 


ST.DEV. 




MEAN 


+1SD 


-1SD 


ST.DEV. 


1993 


Disabled 


-1.1441 


-0.1824 


-2.1059 


0.9618 




-1.1714 


-0.2489 


-2.0939 


0.9225 


1993 


General 


0.0205 


0.9911 


-0.9501 


0.9706 




0.0312 


0.9764 


-0.9140 


0.9452 


1993 


Total 


-0.0635 


0.9522 


-1.0792 


1.0157 




-0.0555 


0.9380 


-1.0491 


0.9935 


1994 


Disabled 


-0.9608 


0.0526 


-1.9743 


1.0134 




-0.5452 


0.1277 


-1.2182 


0.6730 


1994 


General 


0.3526 


1.3085 


-0.6032 


0.9558 




0.2913 


1.0526 


-0.4699 


0.7613 


1994 


Total 


0.2418 


1.2697 


-0.7860 


1.0279 




0.2207 


1.0100 


-0.5685 


0.7892 


1995 


Disabled 


-0.5092 


0.3391 


-1.3575 


0.8483 




-0.2362 


0.5344 


-1.0068 


0.7706 


1995 


General 


0.3591 


1.1392 


-0.4210 


0.7801 




0.5185 


1.2808 


-0.2438 


0.7623 


1995 


Total 


0.2900 


1.1101 


-0.5301 


0.8201 




0.4584 


1.2483 


-0.3314 


0.7898 


1996 


Disabled 


-0.4409 


0.3143 


-1.1960 


0.7551 




-0.1678 


0.4922 


-0.8279 


0.6601 


1996 


General 


0.3834 


1.1063 


-0.3394 


0.7228 




0.5044 


1.2122 


-0.2035 


0.7078 


1996 


Total 


0.3171 


1.0764 


-0.4423 


0.7594 




0.4502 


1.1777 


-0.2772 


0.7275 








SCIENCE 








SOCIAL STUDIES 








MEAN 


+1SD 


-1SD 


ST.DEV. 




MEAN 


+1SD 


-1SD 


ST.DEV. 


1993 


Disabled 


-1 .0663 


-0.0727 


-2.0600 


0.9937 




-1.0000 


-0.0572 


-1.9428 


0.9428 


1993 


General 


0.0164 


0.9644 


-0.9316 


0.9480 




0.0147 


0.9930 


-0.9637 


0.9784 


1993 


Total 


-0.0617 


0.9301 


-1.0534 


0.9917 




-0.0585 


0.9520 


-1.0690 


1.0105 


1994 


Disabled 


-0.7375 


-0.0407 


-1.4344 


0.6969 




-0.9197 


0.0729 


-1.9123 


0.9926 


1994 


General 


0.0689 


0.8014 


-0.6635 


0.7325 




0.2683 


1.2408 


-0.7041 


0.9724 


1994 


Total 


0.0009 


0.7641 


-0.7623 


0.7632 




0.1681 


1.1967 


-0.8605 


1.0286 


1995 


Disabled 


-0.5628 


0.2269 


-1.3525 


0.7897 




-0.6547 


0.3701 


-1.6795 


1.0248 


1995 


General 


0.1807 


0.9384 


-0.5769 


0.7576 




0.4096 


1.4332 


-0.6141 


1.0236 


1995 


Total 


0.1216 


0.9080 


-0.6648 


0.7864 




0.3249 


1.3883 


-0.7386 


1.0635 


1996 


Disabled 


-0.6646 


-0.0291 


-1.3000 


0.6355 




-0.6533 


0.1492 


-1 .4558 


0.8025 


1996 


General 


-0.0282 


0.6374 


-0.6938 


0.6656 




0.2199 


1.0753 


-0.6356 


0.8555 


1996 


Total 


-0.0795 


0.6060 


-0.7649 


0.6854 




0.1495 


1.0334 


-0.7343 


0.8839 






NUMBER 


PERCENT 


MEAN = 


: Mean 4th grade equated scale score. 




1993 


Disabled 


3564 


7.21% 




+1SD = 


Mean plus one standard deviation. 






1993 


General 


45852 


92.79% 
















1993 


Total 


49416 


100.00% 




-1SD = 1 


Mean minus one standard deviation. 






1994 

1994 


Disabled 

General 


4383 

47557 


8.44% 

91.56% 




ST. DEV. = Standard deviation of equated scale score. 




1994 


Total 


51940 


100.00% 




M.fVii 1 nl 


► e* arc r'nnetan 


if ar'mee 


to r*\f 1 Afitkt i r*\ 


each year. 




1995 

1995 


Disabled 

General 


4075 

47123 


7.96% 

92.04% 






is arc uunsian 


ll abiUSS bun 


leni wunin 


















1995 


Total 


51198 


100.00% 














1996 


Disabled 


4083 


8.05% 














1996 


General 


46609 


91.95% 














1996 


Total 


50692 


100.00% 
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Appendix C. Grade 11 Students with Disabilities Compared to General Population: Mean 
Equated Scale Score, Standard Deviations, and Counts 



YEAR 


MEAN 


READING 

+1SD -1SD 


ST.DEV. 




MEAN 


MATHEMATICS 

+1SD -1SD 


ST.DEV. 


1993 


Disabled 


-1.379716 


-0.4398 


-2.3196 


0.939870 




-0.932775 


-0.0040 


-1.8616 


0.928791 


1993 


General 


-0.037231 


0.9478 


-1.0223 


0.985048 




-0.057188 


0.9353 


-1.0497 


0.992534 


1993 


Total 


-0.086988 


0.9286 


-1.1026 


1.015577 




-0.089641 


0.9143 


-1.0936 


1.003953 


1994 


Disabled 


-1.002825 


0.0163 


-2.0220 


1.019125 




-0.774055 


-0.0717 


-1.4764 


0.702353 


1994 


General 


0.470197 


1.4338 


-0.4934 


0.963644 




0.263299 


1.1274 


-0.6008 


0.864107 


1994 


Total 


0.401413 


1.4165 


-0.6136 


1.015042 




0.214859 


1.0996 


-0.6699 


0.884726 


1995 


Disabled 


-0.818687 


0.1208 


-1.7581 


0.939443 




-0.431224 


0.3067 


-1.1691 


0.737923 


1995 


General 


0.479224 


1.3859 


-0.4274 


0.906667 




0.460446 


1.2879 


-0.3670 


0.827485 


1995 


Total 


0.415842 


1.3662 


-0.5345 


0.950383 




0.416902 


1.2624 


-0.4286 


0.845460 


1996 


Disabled 


-0.628760 


0.2126 


-1.4701 


0.841385 




-0.444211 


0.2350 


-1.1235 


0.679250 


1996 


General 


0.457967 


1.3469 


-0.4310 


0.888952 




0.431742 


1.2336 


-0.3701 


0.801844 


1996 


Total 


0.371846 


1.3316 


-0.5879 


0.959758 




0.366817 


1.2038 


-0.4701 


0.836966 








SCIENCE 








SOCIAL STUDIES 








MEAN 


+1SD 


-1SD 


ST.DEV. 




MEAN 


+1SD 


-1SD 


ST.DEV. 


1993 


Disabled 


-1.153781 


-0.1495 


-2.1581 


1.004306 




-1.151463 


-0.1797 


-2.1233 


0.971788 


1993 


General 


-0.066788 


0.9592 


-1.0928 


1.025970 




-0.087502 


0.9657 


-1.1407 


1 .053244 


1993 


Total 


-0.107076 


0.9385 


-1.1526 


1.045527 




-0.126936 


0.9424 


-1.1963 


1 .069385 


1994 


Disabled 


-0.897775 


-0.0465 


-1.7490 


0.851252 




-0.794736 


0.1613 


-1.7508 


0.956048 


1994 


General 


0.333904 


1.2524 


-0.5846 


0.918528 




0.518427 


1.5135 


-0.4766 


0.995028 


1994 


Total 


0.276389 


1.2280 


-0.6753 


0.951655 




0.457108 


1.4883 


-0.5740 


1.031149 


1995 


Disabled 


-0.623305 


0.2529 


-1.4995 


0.876220 




-0.721593 


0.2242 


-1.6673 


0.945752 


1995 


General 


0.519700 


1.4084 


-0.3690 


0.888745 




0.501464 


1.5076 


-0.5046 


1.006106 


1995 


Total 


0.463882 


1.3855 


-0.4578 


0.921659 




0.441737 


1.4790 


-0.5955 


1.037285 


1996 


Disabled 


-0.475787 


0.2739 


-1.2255 


0.749670 




-0.724017 


0.1473 


-1.5954 


0.871338 


1996 


General 


0.505754 


1.3484 


-0.3369 


0.842627 




0.325053 


1.2725 


-0.6224 


0.947467 


1996 


Total 


0.433107 


1.3194 


-0.4532 


0.886332 




0.246161 


1.2410 


-0.7487 


0.994830 







NUMBER 


PERCENT 


1993 


Disabled 


1336 


3.71% 


1993 


General 


34710 


96.29% 


1993 


Total 


36046 


100.00% 


1994 


Disabled 


1710 


4.67% 


1994 


General 


34910 


95.33% 


1994 


Total 


36620 


100.00% 


1995 


Disabled 


2017 


4.88% 


1995 


General 


39286 


95.12% 


1995 


Total 


41303 


100.00% 


1996 


Disabled 


2080 


4.96% 


1996 


General 


39844 


95.04% 


1996 


Total 


41924 


100.00% 



MEAN = Mean 4th grade equated scale score. 

+1SD = Mean plus one standard deviation. 

-1SD = Mean minus one standard deviation. 

ST.DEV. = Standard deviation of equated scale score. 
N-Counts are constant across content within each year. 
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Appendix D. Students with Disabilities and Accommodations: Mean Equated Scale Scores 1994-95 and 1995-96 





Q 


3 


m 


in 


CO 




o 


CM 


0 


0 




in 


O 


CO 


CM 


CO 


t- 


in 


0 


CO 


NT 


in 


CO 


05 


Y- 


0 


CM 


05 


I s - 


co 


co 


CO 


CM 


M- 


CO 


CO 


CO 


O 


CO 




CO 


s 


CO 


h- 


CO 


CO 


CM 


CO 


0 


I s - 


CM 


CM 


CM 


CM 


r- 


CM 


co 


CO 


0 


3 


3 


TJ- 


CO 


Y— 


CO 


O 


r- 


co 


CO 


I s - 


CO 


3 


O 


O' 


co 


05 


O 


Y— 




CD 


in 


m 


in 




3 


CO 


0 


CO 


CO 


I s - 




CO 


m 


I s - 


CO 


CM 


CO 


I s - 


CO 


Y“ 


0 


CO 




CO 


CO 


CO 


CM 


00 


05 


0 


I s - 


in 


Y— 






h- 


y— 


o 


o 


o 


o 


I s - 


05 


CO 


CO 


CO 


I s - 


CO 


CO 


CO 


p 


p 


p 


05 


05 


p 


CO 


CO 


I s - 


CO 


I s - 


I s - 


CO 


0 


05 


CO 


CO 


p 


p 


h- 


h- 


CO 






o 














d 


O 


d 


d 


d 


d 


O 


d 


d 






d 


d 


d 




d 


d 


d 


d 


d 


d 


d 




d 


d 


d 


d 


d 


d 


d 


d 


CO 


o 


I s - 


CO 


CO 


05 


CO 


CO 


05 


CO 


CO 


1 s - 


CO 


3 


CO 


05 


3 


CO 


co 


co 


CO 






I s - 


CO 


05 


Y_ 


co 


05 


NT 


NT 


in 


I s - 


NT 


0 


I s - 


co 


co 


co 


Y_ 


LU 


CO 


o 


CO 




3 


CO 


CM 


05 


0 


CO 


CM 


CO 


05 


CM 


CM 


CO 


in 


in 


05 


CO 


in 


3 


in 


3 


CO 


co 


CM 


05 


in 


0 


00 


co 


co 


CO 


co 


co 


3 




o 




CO 


Y— 


■O' 


CM 


in 


O 


I s - 


TT 


CM 


T— 


in 


0 


CO 


CO 


05 


CO 


CO 


0 


05 


CM 


I s - 


05 


3 


GO 


05 


CM 


in 


in 


O' 


Tj- 




co 


Li 


T— 


CM 


in 


in 


I s - 


Y— 


CM 


CO 


CM 


T— 


CM 




CO 


CO 


05 


in 




I s - 


I s -; 


I s - 


in 


p 


CM 


I s - 


in 


in 


CM 


7 


Y— 


in 


I s - 


cq 


in 


I s - 


p 


in 


p 


3 


1 


o 

i 


V 

1 


V 

1 


o 


V 

1 


V 

1 


o 

i 


d 


V 

1 


9 


T— 


d 

1 


d 


d 


d 


d 


d 

• 


T— 

1 


T— 


T— 

1 


T— 

1 


Y— 

1 


d 

1 


Y— 

1 


Y— 


Y— 


Y— 


Y— 

1 


Y— 


d 


Y— 


Y— 


Y— 


d 


Y— 


Y~ 


Y~ 


Y— 

1 


CO 


Q 


o 


CO 


CM 


I s - 


CO 


■O' 


CO 


CO 


CO 


in 


CO 


CO 


CO 


in 


CO 


CO 


CO 


CO 


CO 


CO 


0 


in 


3 


CM 


0 




O 


O 


CO 


0 


0 


O 


I s - 


0 




CO 


I s - 






CO 


CO 


r- 


y— 


05 


CO 


05 


3 


U) 


CO 




CO 


0 


3 






CM 


CO 


CO 


3 


05 


CO 


co 


CO 


CM 


CM 


CO 




I s - 


05 


I s - 


O 


h- 


Y — 


0 


CO 


co 


05 


s 


CO 


CO 


in 


U) 


05 


CO 




05 


0 


^ — 


CM 


▼— 




CO 


CO 


in 


in 


CO 


m 


CO 


05 


CM 


CO 


co 


CO 


CO 


I s - 


co 


CO 


O 


O' 


h- 


CO 


CO 


CO 


o 


— 


CO 


I s - 


in 


CO 


05 


I s - 


■M" 


CM 


CO 


m 


in 




CO 


I s — 


CM 


CM 


CO 


CM 




CO 


CM 


co 


p 


O 




CO 


0 


0 


in 


7 


Y — 


0 


CM 


CM 




0 


0 


O 


o 

CO 


+ 




o 


o 




O 


o 




^ 


d 




d 




d 


d 








d 


d 


d 


d 


d 




d 


d 


d 


d 

1 


d 


d 




d 


d 


d 


Y— 


d 


d 


d 


d 


z 


CD 


CO 


CO 


Tt 


CO 


CO 


■O' 


CM 


CO 




r- 


CO 


in 


r— 


CM 


05 


05 


co 


in 


00 


in 




in 


CO 






0 


I s - 


co 


I s - 


05 


CM 


I s - 


CM 


05 


3 


r- 


in 




rl* 


00 


CM 


m 


CM 


I s - 


CO 


CM 


CM 


CO 


TT 


in 


CO 


I s - 


0 


3 


05 


■o- 


05 


in 


3 


— 




05 


TT 


CO 


CO 




in 


0 






Nf 


CM 


CO 


I s - 


co 


CM 




Ml 


CO 


CO 


o 


o 


m 


CM 




CO 


0 






rr 


I s - 


CM 


O 


CM 


CO 


0 


CD 


CM 


Tr 




CM 


3 


CO 


0 


0 


3 


05 




r- 


O' 


co 


CO 


CO 


I s - 






LO 


CO 


m 


CO 


o 


CM 


■O' 


'O’ 


CM 


CO 


CO 


CM 


0 


T— 


CO 




CO 


I s - 


CO 


CO 


CO 


CM 




I s - 


CO 


I s - 


I s - 


co 


I s - 


CO 


co 


CM 


h- 


CO 


CO 


h- 




2 


o 


o 


o 


o 


o 


o 


O 


O 


d 

1 


d 


0 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


9 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 

1 


d 


d 

1 


d 

1 




Q 


3 


CO 


T— 


3 


05 


r- 


O 


O 


■<fr 


t— 


-v— 


CM 


I s - 


CM 


05 


T— 


3 


I s - 


CO 


CM 


CO 


05 


3 


I s - 


CM 


r— 


0 


0 


in 


I s - 


0 


Y— 


Y— 


co 


in 


Nh 


■O' 


co 




CO 


CO 


CM 


3 


CO 




CO 


0 


in 


3 


Of) 


CM 


0 


O 


3 


05 


0 


O 


CO 


O 




0 


r— 


05 


co 


05 




I s - 




(O 


co 


I s - 




I s - 


0 




CD 


I s - 


CO 




CM 


CO 


CO 


05 


I s - 






0 




CO 


05 


CO 


CO 


CM 


r— 


CO 


3 


05 


0 


CO 


05 


CO 


CM 


CO 


CM 05 


co 




3 


o - 


0 






I s - 


CO 


CO 


CO 


r- 


CO 


r- 


I s - 


I s - 


I s - 


I s - 


I s - 


I s - 


CO 


I s - 


I s - 


I s - 


I s - 


I s - 


I s - 


I s - 


r— 


CO 


in 


CO 


in 


m 


co 


05 


CO 


CO 


I s - 


co 


co 


CO 


h- 






o 


o 


o 


o 


o 


o 


o 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


O 


d 


d 


d 


d 


d 


d 




Q 


I s - 


CO 


CO 


I s - 


CM 


CO 


in 


CO 


CM 


I s - 


0 


in 


CO 


05 


in 


r— 


CO 


CM 


CM 






co 


05 


■O’ 


M- 


I s - 


M- 


Y— 


CM 


CO 


CO 


O 


05 


CM 


co 


I s - 


in 


0 




CO 


o 


in 




CO 


CO 


3 


T— 


0 


CM 


m 


in 


r- 


r— 


CO 




r— 


3 


CO 


— 




in 


Y 


3 


0 




3 


CM 


CM 


Y— 


h- 


3 


Tt 


05 


CO 


I s - 


00 


0 


h- 




o 


in 


Nf 




05 


r- 


in 


CO 




05 




I s - 


I s - 


CM 


in 


CO 


05 


in 


LO 


05 


0 


CO 


05 


CO 


in 


in 


I s - 


co 


in 


h- 




05 




LLI 




CM 


h- 


h- 


y— 


CM 


■o- 


o 


CM 


I s - 


■«— 


I s - 




m 


r— 


CO 


CM 


CO 


NT 


CO 


co 


CM 


05 


r— 


NT 


CO 




CM 


CM 


p 


n- 


in 


in 


CO 


O' 


Csl 


co 




CM 


O 


* 


o 


o 


o 


o 


o 


o 


o 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


v 


v 


V 


v 


O 


d 


' 7 ’ 


7 


7 


7 


7 


7 


d 


7 




7 


d 


1 


1 


1 


1 


LU 


Q 


o 


■o- 


in 


CM 


CO 


CO 


in 


CO 


CO 


in 


CM 


0 


CO 


in 


CO 


in 


0 


CO 


CO 


0 




O 


0 






CO 


3 


CM 


co 


m 




CO 


CO 


O' 


CO 


0 


3 


^ 


O 


CO 


CO 




CM 


CM 


CO 


CM 


CO 


m 


CO 


3 


CO 


05 


I s - 


CO 


r- 


0 


CO 


CO 


0 


05 


CM 


O 


CO 


r— 


Y — 


0 


0 


I s - 


in 


05 


CO 


co 


05 


I s - 


8 


3 


CO 


CO 


o 


CO 


CO 


^ — 


CO 


05 


^ — 


■*fr 


05 


CO 


3 


CM 


m 


1 s - 


0 


— 


r- 


CO 


05 


m 


0 




00 


r— 




05 


00 


CO 


0 


co 




Y - 


in 


0 


y— 


CO 


o 


05 


in 


CM 


y— 


m 


CM 


CO 




CO 


0 


CO 


■»- 


CM 


05 


T— 


0 




T— 


LO 


CO 


Y-“ 


Y— 


0 


▼— 


O 


CM 


CO 


CM 


0 


CM 


CO 


co 


O 




T™ 




+ 


*■" 




o 










T_ " 


d 




d 




d 


d 






d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


O 


d 


d 




d 


d 


d 


Y~ 


d 


d 


d 


d 




z 


CO 


o 


in 


00 


r- 


05 


m 


CM 


CM 


3 


05 


r— 




CO 


3 


3 


CO 


3 


3 


CM 


m 


05 


in 


1 s - 


CO 




3 


Y— 


r- 


05 


CO 


05 


00 


Y - 


co 


co 


Y - 


3 




<< 


CO 


CO 


o 


CM 




CO 


CM 


CM 


00 


0 


in 


m 


CO 






T— 


O 


05 


co 




r- 


CO 




s 


I s - 


CM 


CO 


CO 


05 


r- 


CO 




Ml 


CO 


CM 


CM 


CO 


CO 


in 


CO 


CO 


in 


in 


in 


CM 


CO 


CM 


*— 


O 


CM 


CO 


(O 


05 


CO 


CM 


I s - 


in 


I s - 


co 


O 


00 


co 


r- 


in 


h- 


CO 


in 


h- 


3 








to 


y— 


y— 


h- 




CO 


CO 


xr 


0 


CO 


0 


CO 




0 




m 


*— 


CO 


CO 


in 


m 


CM 


O 


r— 


I s - 


in 


I s - 


CO 


co 


M- 


co 


h- 


in 


O' 




co 


LO 




a 


o 


o 


o 


o 


o 


o 


o 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 

• 


d 

• 


d 

• 


d 


d 

• 


d 


d 


d 


d 


d 


d 

• 


d 


d 


d 


d 


d 


d 


d 


d 

1 


d 


d 




Q 


Tt* 


o 


in 


I s - 


CO 


CO 


I s - 


05 


3 


CM 


■M- 


CO 


3 


CO 


•v— 


r- 


CO 


CO 


CO 


co 


r- 


CM 


m 


3 


CO 


Y— 


co 


8 


0 


in 


M- 


Y— 


00 


0 


0 


in 


CM 


0 




CO 


I s - 


I s - 


CO 


I s - 


in 






0 


in 




05 


05 


3 


CO 


05 


m 




3 


m 


CO 


r— 


I s - 


05 


M- 


I s - 


in 


CM 


CO 


in 


I s - 


0 


h- 


h- 






CM 


CO 


in 


CO 


in 


in 


3 


05 


CM 


1 s - 


0 


in 




CM 


3 


CO 


co 


CO 


m 


05 


CM 




co 


CO 


CM 


M- 




3 




TT 


h- 


co 


CM 


CO 


00 


0 






h- 


h- 


h- 


CO 


CO 


h- 


CO 


h- 


CO 




r- 


r— 


r— 


r- 


I s - 


1 s - 


CO 


I s - 


r— 


I s - 


I s - 


co 


in 


I s - 


CO 


CO 


I s - 


I s - 


CO 


co 


CO 


I s - 


in 


in 


co 






o 


o 


o 


d 


o 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


CO 


O 


CM 


3 




00 


I s - 


CO 


I s - 


0 


0 


05 


CM 


CM 


0 


CO 


r— 


co 


■*3* 


in 


r— 


co 


05 




CM 


co 


CO 


0 


0 


CM 


05 


co 


CO 


05 


co 




co 




co 


CM 


CO 




CO 




CM 


CO 


05 


CM 


CM 


3 


00 


<0 


r— 


3 


CM 


CM 


y— 


co 


r- 


CO 


00 


CO 


1^ 


co 


in 


M- 




CO 


in 


00 


05 


in 


CO 


0 


in 


CM 


I s - 


CO 


o 




O 


in 


h- 


in 


r- 


CO 




CO 


I s - 


CO 


CO 


00 


’M- 


CO 


0 


05 


T— 


in 


05 


I s - 


05 


co 




in 


CO 


co 


CM 


O 


CO 


05 


h- 


h- 


CO 


■0“ 


CO 






Y— 


CO 


CO 




m 


■<3- 


CM 


■*— 


in 


O 


-M- 


CO 


O’ 


M- 


CM 


CM 


CO 


T— 


05 


p 


05 


CO 


CM 


00 


co 


5 


co 


I s - 


in 


Nf 


CO 


p 


05 


o- 


CM 


Y— 




05 


< 


1 


o 

1 


d 

1 


d 

i 


d 

i 


o 

i 


o 

i 


d 

i 


O 

1 


d 

1 


d 


d 

1 


d 

1 


d 

1 


d 

1 


d 

1 


O 

1 


d 

1 


T 


d 

1 


T 


d 

• 


d 

1 


d 

1 


d 


d 

1 


d 

1 


d 

1 


d 

1 


d 

1 


d 

1 


7 


7 


d 

1 


d 

1 


7 


7 


7 


d 

1 


s 

UJ 


Q 


in 


CO 


05 


in 


05 


o 


I s - 


CO 


CO 


3 


CO 


CO 


00 


05 


in 




CO 




3 


CO 


in 


CO 


I s - 


CM 


I s - 


CM 


CM 


I s - 




Tf 


0 


CM 


I s - 


co 


I s - 


0 


CM 


CO 


x 


CO 


CO 


Nf 


CO 


CO 


CO 


o 


CO 


in 


CO 




CO 


CO 


CO 


in 


in 


CO 


3 


0 


CM 


05 


I s - 


r- 


05 


3 


co 


05 


CO 


CM 


in 


O 


h- 


co 


3 


co 


I s - 


CO 






CM 


in 


o 


in 


CO 


05 








CM 


in 


05 


0 


CO 


in 


Nf 


CO 


in 


m 


05 


I s - 


CM 


TT 


05 


05 


0 


CO 


Y*“ 


O 


in 


0 




CM 


CO 


< 

s 


y— 


CO 


05 


CO 


CM 


T— 


o 


CO 


CM 


05 




05 


T— 


05 


p 


CM 




CM 


co 


CO 


in 


in 


CO 




CO 


CM 


CO 


CO 


Nf 


05 


CM 


Y— 


CM 


co 


CM 




0 


O 


CM 


+ 




O 


o 






T ” 


T_ 


T_ 


d 


T_ 


O 




d 




T “' 






d 


d 


d 


d 


d 




d 


d 


d 


d 


d 


d 




d 


d 


d 




d 


d 


d 


d 




z 


y- 


CO 


3 


05 




CM 


o 


05 


3 


CM 


CM 


in 


3 


f— 




3 


3 


CM 


CM 


r— 


CM 


-r- 


CM 


CM 


O 


05 


3 


co 


Y_ 


05 


■yj- 


CO 


Y_ 


00 


co 


co 




CM 




< 
1 1 1 


CO 


h- 


in 


CO 


00 


CM 




O 


CO 


CO 


NT 






CO 


co 


co 


CO 


O 


CO 


00 


M- 


co 




CO 


I s - 


CM 


CO 


CO 


in 




0 


I s - 




00 


in 


O 


CO 


o 


I s - 


m 


in 


05 




CM 


05 


r- 


I s - 


O 


0 


in 


r— 


T— 


co 


0 


0 


in 


00 




I s - 


CM 


M- 


CO 




05 


3 


Y“ 


CO 


in 


h- 


CO 


co 




UJ 


in 


T— 


T— 


CO 


CO 


CM 


m 


in 


T— 


I s - 


CM 


CO 


CM 


CM 


in 


CD 


tj- 


co 


CO 


CM 


CM 




Nf 


CM 


CO 


O 


CM 




Y— 


Nf 


in 


CO 


CO 


in 


in 


in 


CO 




2 


d 


o 


o 


o 


o 


o 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 

• 


d 

t 


d 

• 


d 


d 


d 


d 

• 


d 


d 


d 


d 

T 


d 


d 


d 


d 


d 


d 


d 


d 

• 


d 


d 




o 


05 


CO 


CO 


I s - 


■<T 


CO 


CO 


0 


05 


r- 


3 


CO 


CM 


in 


CO 


T— 


T— 


05 


T— 


00 


r- 


I s - 


NT 


Y— 


CO 


I s - 


CO 




I s - 




Nf 


I s - 


Y— 


CO 


in 


in 


co 


CO 




CO 


3 


05 


i^ 


CO 




CM 


o 


CO 


CO 




0 - 




CM 




r- 


O 




I s - 


co 


co 


CN 


05 


I s - 


in 


co 


CO 


05 




0 


r- 




O 


05 


co 


in 


co 


in 




CO 


CM 


I s - 


CO 


in 


CO 


CO 


I s - 


r- 


3 


05 


<0 


CO 


CO 


T— 


CM 


05 


0 


05 


r— 


m 


in 


05 


0 


05 




O 


CM 


in 


co 




3 


in 


co 


Y“ 


co 


in 






CO 


05 


05 


CO 


CO 


00 


CO 


CO 


CO 


r— 


CO 


CO 


CO 


CO 


05 


CO 


CO 


CO 


I s - 


I s - 


CO 


I s - 


I— 


r— 


I s - 


I s - 


I s - 


I s - 


05 


05 


co 


05 


05 


I s - 


h- 


h- 






o 


o 


o 


o 


d 


o 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 




Q 




3 


o 




I s - 


05 


CO 


CO 


^r 


CO 


r- 


CO 


05 


I s - 


CM 


CO 




r— 


CO 


CM 


in 




co 


CO 


co 


Y— 


Y_ 


05 


co 


in 


in 


I s - 


3 


05 


05 


in 


CM 


co 




CO 


CO 


CO 


in 


CO 


o 


CO 


CO 


T— 


CO 


I s - 


I s - 


0 


05 


0 


CO 


0 


0 




CM 


co 


CO 


CM 


0 


co 


Y 


05 




co 


3 


CM 


r*- 


I s - 


0 


CM 


CO 


05 




CO 


CO 


h- 


in 


CO 


CO 


o 


CM 


3 


r— 


^r 


05 


0 


in 




CM 


CO 


05 


05 


CO 


r— 


CO 




0 


in 


05 


CM 


CO 


co 


CM 


I s - 


co 


co 


05 


0 


0 


CO 


o 

z 


Y— 


o 


I s - 


r- 


o 


”0- 


in 


T— 


0 


CM 


in 


T— 


in 


m- 


0 




in 




CO 


CM 


CM 


0 


NT 


CO 


CM 


Y— 


CM 


Y— 


CO 


in 


CO 


co 


p 


in 


in 


in 


in 


CO 


1 


d 

i 


o 

i 


o 

i 


d 


d 

i 


d 

i 


o 


d 

1 


d 


d 


d 


d 


d 


d 


d 


d 


d 












d 












d 


d 


















Q 

rs 


o 


r- 


CM 


CM 


CM 


T 


CO 


CO 


CO 


CO 


CM 


CM 




m 


3 


0 


3 




O 


CO 


3 


05 


CO 


3 


05 


co 


3 


■<3- 


0 


05 


CM 


CM 


CO 


I s - 


co 




in 


in 


CO 


UJ 


CO 




CO 


CM 


CM 


3 




05 


I s - 


CO 


r— 


m 


▼“ 


r- 


CO 


O 


CO 


05 


CO 


05 


CO 


3 


I s - 


I s - 


05 


co 


CM 


3 


3 




co 


Y— 


05 




X 


3 


in 


00 


O) 


I s - 


CO 


3 






NT 


05 


co 


0 


CM 


^ — 




05 


T— 


O 


I s - 


Y — 


I s — 


05 


05 


O 


co 


I s - 


co 




co 


I s - 


I s - 


CM 


CM 




Y— 


■»— 


O 


CO 


CM 


t- 


I s - 


CM 


co 


1— 


in 


*— 


CO 


r— 


r- 




CM 


CM 


CO 


CM 


CD 


0 


CM 


Y— 


CO 


CM 


CM 


in 


co 


Y“ 


Y— 


O 


co 


CM 


O 


0 






+ 


T_ 










*- 










T ~ 














d 


d 


d 


d 


d 




d 


d 


d 


d 


d 


d 




d 


d 

• 


d 




d 


d 

• 


d 


d 




z 


CO 


05 


CO 


in 


I s - 


CO 


05 






in 


00 


CO 


CO 


05 




co 


0 


CO 


CO 


3 


CO 


3 




CM 


0 


■^r 


CO 


Tf 


CO 


CO 




0 


3 


00 


tj- 


0 


co 


0 






CO 


CO 


3 


CO 


CM 




CO 


05 


r- 


m 


CO 


CO 


CO 


CM 




CO 


0 


CO 


I s - 


05 


r- 


CO 




I s - 


0 


CM 




in 


in 


co 




h- 


r- 


r- 


3 




Ml 


h- 


CO 




CM 


CM 


CO 


O 


co 


3 


05 


05 


CO 


CM 


3 


05 


05 


05 


co 


CO 


05 


CO 




0 


in 


05 




I s - 


Nf 




in 


0 


05 


hi 


in 


00 


co 




LU 


r- 


T— 


T— 


5 


'O' 


CO 


05 


CO 


CO 


CM 


CO 


CO 


'O' 


I s - 


CM 


in 


in 




■< 3 - 


CM 


co 


in 


in 


CO 


in 






-o- 


CO 


p 


I s - 


CO 


co 


I s - 


I s - 


CO 




a 


o 


o 


o 


d 


d 


o 


d 


d 


d 




O 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 


d 

1 


d 


d 


d 

• 


d 

• 


d 

1 


d 

1 


9 


d 


d 

1 


7 


d 

1 


d 


9 


9 


d 

1 


d 

1 




o 


io 


r— 


M- 


r- 


o 


r- 




O) 


<0 


IN 


ID 


r- 


CN 


r— 


0 


ID 


00 


0 


r- 


00 


D 


t- 


CN 


ID 


1— 


0 


CO 


CO 


CO 


CO 


ID 


D 


CO 


Yt 


CN 


Y— 


0 


0 






CM 


o 


CN 


<d 


CO 






ID 


M- 


CN 


r— 


§ 


CN 




CN 


O 


05 


CN 


05 


8 


O 


CN 


CD 


r— 


8 


CD 


ID 


CN 




0 


3 


O 


ID 


CN 


CT> 


<D 


r- 


B 




ct 


S 


o 


id 


oo 


Y“ 


r- 


cn 


r- 


r- 




CO 


CN 


ID 


co 


*-* 


I- 


N- 


ID 


*- 


C 5 


CO 


CN 




CO 


0 


NT 


CO 


CN 


ID 


o> 


r— 




CN 




u_ 


M- 














m- 


















ID 












§ 


































Z 

o 














X 

1- 
















O 


O 

Q 

06 

H 

O 

LU 












0 














O 






















H 














0 
















O 


H 












Q 














Q 






















< 














06 
















06 


06 












06 














o0 






















Q 












_l 
















_l 


—l 


_l 












_l 




































o 

2 








o 




£ 


£ 










O 




£ 


£ 


£ 










££ 








O 




£ 


£ 








£ 










£ 










Q 




o 


0 










Q 




O 


O 


O 










O 


0 








Q 




O 


O 








O 










O 




2 


-J 


LU 




oO 




o 0 


06 


_l 


lil 






06 




06 


°5 


°5 




LU 






oO 






LU 




06 




oO 


oO 


_l 


hi 




oO 


_l 


ill 






ofi 




o 

o 

g 


< 

H 

o 


Z 

o 

z 


—1 _l 
££ 


££ 
< < 


£ 

< 


< 

H 

O 


Z 

O O 

-y — 


t 


_l 

£ 


££ 
< < 


£ 

< 


£ 

< 


< z 
o° 


_J 

£ 


£ £ 
< < 


£ 

< 


< z 
0° 


_j —I 
££ 


££ 
< < 


£ 

< 


< z 


££ 
< < 


< z 

5° 


£ 


££ 
< < 




< 


h- 




o 


O CL CL 


CL 


\- 


• 


Q O O £L CL 


CL 


CL 


H 


• 


O 


CL 


CL 


CL 


H 


i 


O 


O 


CL CL 


CL 


H 


> 


CL CL 


H 




O 


CL 


CL 






in 


in 


in 


in 


in 


in 


in 


co 


CO 


CO 


CO 


CO 


CO 


CO 


CO 


CO 


in 


in 


in 


in 


in 


in 


co 


CO 


co 


co 


CO 


CO 


CO 


in 


in 


in 


in 


co 


CO 


co 


CO 


CO 




(V 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 




LL 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 


05 05 




>- 


T— 


T— 


T— 


T— 


T— 


T— 


T— 


T— 


T— 


T— 


T— 


T— 


T— 


T— 




T— 


■»— 


T— 


T— 


■»— 


Y— 


Y— 


Y— 


Y— 


Y— 


Y— 


Y— 


Y— 


Y— 


T— 


Y~ 


Y- 


Y- 


Y- 


Y~ 


Y~ 








d 

ac 

CD 


333333 


1 04 


04 

04 

04 

04 

04 

04 

04 


1 04 


3 


CO 

0 


08 


08 


CO 

0 


08 


GO 

0 


GO 

O 


GO 

0 


GO 

O 


08 


08 


08 


80] 


Y— 


Y“ 


Y“ 


Y“ 




Y“ 


Y“ 







46 



NCEO 



BEST COPY AVAILABLE 



NATIONAL 

CENTER ON 
EDUCATIONAL 
O U T COMES 

University of Minnesota • 350 Elliot Hall 
75 East River Road • Minneapolis, MN 55455 
612.624.8561 • Fax 612.624.0879 
http://www.coled.umn.edu/NCEO 




The College of Education 
&. Human Development 

University of Minnesota 




. , . 49 

er|c copy available 








US. DEPARTMENT OF EDUCATION 

Office of Educational Research and Improvement ( OERl ) 
Educational Resources Information Center (ERIC) 




NOTICE 



reproduction basis 




Tills document is covered by a signed “Reproduction Release 
(Blanket)” form (on Hie within the ERIC system), encompassing all 
or classes of documents from its source organization and, therefore, 
docs not require a “Specific Document Release form. 



This document is Federally-funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may 
be reproduced by ERIC without a signed Reproduction Release 
form (either “Specific Document or Blanket ). 



3 

ERIC 



wo\ 




